It is well known that large rooms are a challenging environment for acoustic communication. For example, in a typical classroom the distance between the teacher and the students is considerably larger than the typical distance of about 1 to 1.5 meter during a normal one-to-one communication. Therefore, the voice of the teacher is relatively weak at the position of the student. External noise sources as well as the voices or other sounds coming from fellow students add to the acoustics, resulting in a low or even negative signal-to-noise ratio. In addition, the walls of the room add acoustic reverberation to the voice of the teacher, which further reduces its intelligibility.
While the above problems are especially severe for hearing impaired students in a classroom, they are also present outside the classroom, e.g. during a business meeting in a large room, in a room with poor room acoustics or simply because some meeting participants talk or make sounds during a presentation.
In the prior art solutions are known implemented based on radio transmission of the teacher's voice to the hearing aids of the student. This system is referred to as an ‘FM system’. FIG. 1 provides an illustration. By means of FM wireless transmission, audio data is transferred to a receiver, which then plays the signal to a headphone or hearing aid.
In recent years FM systems have been improved by the implementation of signal-to-noise ratio estimation in separate frequency regions at the teacher position and transmitting this information to the FM receiver in order to improve speech intelligibility by adding gain to the voice of the teacher in frequency regions with more noise energy.
An important limitation of FM systems is that they don't allow changing the signal latency. This is especially critical in applications where latency needs to be increased to be synchronous with, for example, a video stream. Another important limitation is that FM systems are analog, offering no way at the end point to perform signal error correction, and that they are susceptible to interference. Those systems are also unidirectional, making it impossible to transmit back audio from the receiver using the same frequency band.
An important challenge in any digital audio solution is to keep the latency introduced by the digital signal processing as low as possible. The latency of a system is defined as the time difference between the time at which some data is received in the system and the time at which the same data is outputted. The challenge imposed by WiFi technology when it comes to low latency audio, is well known. To the best of our knowledge, no audio-over-network solution available on the market offers an audio latency over WiFi of less than 100 ms. Impairments like jitter, radio mode change or other are very detrimental when it comes to a low latency stream of data.
The main solution to deal with those impairments involves buffering large enough amounts of data before playing the stream out. In this way, the system can withhold periods of time where no data is arriving into the system by playing the data that has been already buffered. One specific issue of low latency systems is the inability to react upon impairments in an efficient way, due to the little amount of data that is buffered (because buffering directly translates into added latency). In that sense, low latency streaming systems adopt an ‘optimistic’ approach, where as little as possible data is buffered to compensate for short time impairments (like jitter). This makes those systems especially vulnerable to impairments that can happen over larger periods of time. Examples of such impairments that “sometimes” happen are radio mode changes or other applications running over the same transport link, occupying all the bandwidth from time to time. Therefore, these low latency streaming systems rely much more on audio inferring/repairing mechanisms due to the higher probability of them running out of data to play out. VoIP solutions can be applied over WiFi to obtain a system capable of streaming real time audio over a local area network. Such systems, however, are generally designed to communicate audio over the Internet and have little restrictions on latency requirements, as they normally interconnect people that do not have direct visual contact because they communicate over larger distance and are not in the same room. Because of the large latency, these systems are not generally suitable for use to transmit audio on a latency-constrained environment such as for communication in the same room.
Similar observations can be made with respect to video data. The importance of low latency can be illustrated for the case of a deaf person who wants to follow a conference at which he is physically present and where an additional video stream is broadcasted to an assistive device (like a smartphone, smartglasses, etc.). The same latency constrained environment occurs in a concert, where the audience receives the audio signal directly from the public address system, but multiple video streams are available for those who want to see on their personal communication devices details of the concert they don't want to miss (for example, a video stream exclusively showing the guitar player or a video stream that shows only the singer). Obviously it is important in these cases to keep the latency of the video signals under control, preferably as low as possible.
It is increasingly important that such solutions can be run on personal multipurpose devices (such as smartphones or tablets). Those devices are becoming a central point of communication for the users and they serve as a platform for the development of various extra functionalities, just by running software solutions on said devices. The same applies for wireless communication platforms. The importance of running solutions on widespread transport links (such as WiFi 802.11X) is growing, not just from a cost point of view, but also from a convenience point of view. Those devices are also easily serviceable, even from remote locations.
It is important to note that in those devices solutions exist that fulfil the above requirements, but no solution running on those commodity platforms meets the needs of the described latency-constrained environment. In those environments the dynamic adaptation of the system to the performance available at any particular time suddenly becomes one of the most critical factors to take into account.
Hence, there is a need for a solution to deal with latency constrained environments on personal multipurpose devices.